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Abstract 

Multiple-stage screening is discussed in the context of the prevention literature. Earlier multiple- 
stage screening studies are presented to illustrate different goals and methods employed. A 
multiple-stage screening procedure is described for the identification of depression among high 
school students. Students in Grades 9 - 12 (N = 862) completed a three-stage screening 
procedure. During the first and second stages, students completed a paper-and-pencil self-report 
measure (Mood and Anxiety Symptom Questionnaire); the third stage involved a computer- 
administered clinical interview with the student. Logistical issues regarding implementing a 
multiple-stage screening procedure in schools are presented. The implications of cut-off scores, 
the comorbidity of anxiety and depression, and other issues regarding the implementation of 
multiple-stage screening are discussed. The presentation provides a potentially cost efficient, 
relatively easily implemented technique developed to identify depressed adolescents that can be 
applied to other emotional and behavioral concerns. 
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Introduction 

There is a long history of interest in the prevention of physical and mental health problems in 
the United States (Spaulding & Balch, 1983). Prevention activities in schools are particularly 
salient, given the emphasis on prevention in current national health reform proposals. 
Historically, school psychologists have been interested in prevention, but less directly involved 
(Zins & Forman, 1988), despite the fact that state boards of education often identify prevention 
as an activity that falls within the realm of school psychological services (e.g., Illinois 
Administrative Code, 2000). 

Caplan (1964) proposed one of the more widely accepted conceptualizations of prevention. 
According to Caplan, there are three levels of prevention: primary, secondary, and tertiary. 
Primary prevention emphasizes reducing the incidence of a disorder occurring within a given 
population. Secondary prevention is concerned with the early identification and treatment of 
individuals experiencing problems. Tertiary prevention focuses on rehabilitation and prevention 
of further deterioration for those with serious problems. School psychologists often are involved 
at all three of these levels, spending considerable time in secondary and tertiary prevention 
activities. The focus of this presentation is on a secondary prevention activity, a large-scale 
screening for depression among adolescents. 

Multiple-stage screening has been used to identify students at-risk for behavior disorders and 
antisocial behaviors. For example. Walker and colleagues (1988) developed a procedure, the 
Systematic Screening for Behavior Disorders (SSBD), that relies on teacher input at Stages 1 
(ranking top three intemalizers and externalizers) and 2 (ratings on critical events index and 
combined frequency index). Stage 3 consists of direct observations in the classroom and on the 
playground of students passing through Stage 2. 
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Charlebois et al. (1994) used a different approach in their multiple-stage screening for 
antisocial behaviors. At Stage 1, kindergarten teachers completed the Preschool Behavior 
Questionnaire for each boy at the end of the school year. A subset of boys who scored above the 
70th percentile on the aggressiveness-hyperactivity-distractibility scale was randomly selected 
for Stage 2; mothers also completed the Child Behavior Checklist as part of Stage 1 . Stage 2, 
which occurred 8 months later, consisted of classroom observations and observations of family 
interactions at home and in a laboratory setting. Stage 3 occurred five years after the Stage 1 
assessments. During Stage 3, students who participated in Stage 2 completed the Self-Reported 
Delinquency Questionnaire. The purpose of the Charlebois et al. study was to find early 
indicators that could predict boys who were at risk for developing antisocial behaviors. 

Alfonso and Santandreu (1999) employed a multiple-stage screening procedure in the 
identification of learning and behavior problems. Stage 1 was a conference day for teachers to 
operationally define the types of learning and behavior problems students experience. Stage 2, 
pre-screening of students, consisted of a teacher-generated list of students who may be 
experiencing learning and/or behavioral difficulties. During Stage 3, students’ academic and 
emotional functioning were assessed using standardized measures. Stage 4 consisted of reporting 
results and making recommendations, particularly for students whose performance fell more that 
one standard deviation below the mean on either the academic or behavioral measures. 

Reynolds (1986) described a multiple-stage procedure that was used to screen for depression 
among students. The procedure consisted of three stages: Stagel, an initial large group screening 
with self-report measures; Stage 2, a second screening with the same self-report measures for 
those youngsters who scored at or above a designated cut-off score at the first screening; and 
Stage 3, an individual clinical interview with children who scored at or above the cut-off score at 
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the second screening. Reynolds (1986) developed his multiple-stage screening technique to 
address the “generalized finding of overendorsement of depressive symptomatology in an initial 
assessment” (p. 1 19). When used in screening for depression among school children, the end 
result was a prevalence rate of 7 to 1 2% for the total sample, rather than the initial rate of 1 8 to 
20%. Adopting Reynolds’ procedure, Laurent et al. (1994) reported similar results using the 
multiple-stage procedure to identify anxiety among children; the initial rate was 18%, but the end 
rate was 4 to 6% of the original sample. 

This brief review of multiple-stage screening studies reveals that different procedures have 
been used, depending on the goal of the particular study. Respondents, time frames, and 
techniques have varied across studies. Our study used a multiple-stage screening procedure 
similar to that employed by Reynolds (1986) and Laurent et al. (1994). 

Method 

Participants 

Participants were 862 youngsters in Grades 9-12 from a rural school district in central 
Illinois. 

Instruments 

Mood and Anxiety Symptom Questionnaire (MASQ; Watson & Clark, 1991) is a 90-item 
measure developed to assess constructs associated with the tripartite model of anxiety and 
depression proposed by Clark and Watson (1991). Briefly, the tripartite model addresses 
negative affect (NA; general emotional distress), positive affect (PA; pleasant engagement with 
the environment), and physiological hyperarousal (PH; somatic arousal). According to the 
model, NA is common to both anxiety and depression, and explains the high rate of comorbidity 
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associated with these disorders. Low levels of PA characterize depression, and elevated levels of 
PH characterize anxiety. 

The MASQ has scales that measure aspects of the tripartite model: General Distress; Anxiety 
(1 1 items), General Distress: Depression (12 items). General Distress: Mixed (15 items). 

Anxious Arousal (17 items), and Anhedonic Depression (22 items). The MASQ has 
demonstrated sound psychometric properties with college student and adult samples (Watson, 
Clark, et al., 1995; Watson, Weber, et al., 1995). In our study, we used the Anxious Arousal and 
Anhedonic Depression scales for the screenings at Stages 1 and 2. To our knowledge, this was 
the first time the MASQ has been used with an adolescent sample. 

Diagnostic Interview for Children and Adolescents-IV (DICA-IV; Reich et al., 1997) is a 
computerized diagnostic interview based on the DSM-IV (American Psychiatric Association, 
1994). Participants completed the adolescent version, which is used with individuals between the 
ages of 13 and 18. The computerized format allows students to complete the interview 
individually at their own pace. The DICA-IV is organized around 25 diagnostic areas in which 
an individual may be experiencing problems. Participants in our study were assessed in the 
following nine areas: Oppositional Defiant Disorder, Conduct Disorder, Major Depressive 
Disorder (past and present). Mania or Hypomania, Dysthymic Disorder, Separation Anxiety 
Disorder, Generalized Anxiety Disorder, Posttraumatic Stress Disorder, and Eating Disorders. 
Reich (2000; Reich et al., 1995) demonstrated good reliability and validity with the DICA-IV. 
Procedure 

This study was part of a school-wide screening for depression that was conducted as part of 
the school-based health clinic. The goal of the screening was to identify participants for 
adolescent treatment groups on depression. 
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Stage 1 ■ Teachers were given a packet of MASQs with instructions on how to administer 
them to students in their homeroom. The means and standard deviations for the Anxious Arousal 
and Anhedonic Depression scales of MASQ were calculated. 

Stage 2 . Those students who met the cut-off score on the Anhedonic Depression scale were 
assessed a second time with the MASQ. This assessment occurred in the school auditorium. We 
attempted to have students complete the second administration of the MASQ within two weeks 
of the original administration. 

We also tried to have students who missed the original administration of the MASQ in their 
homerooms complete the measure at this time. Those who completed their first MASQ at this 
time were assessed a second time, again, usually within a two-week period. 

Stage 3 . Those students who continued to obtain scores at or above the cut-off score on the 
Anhedonic Depression scale completed the computer-administered DICA-IV. The DICA-IV was 
scheduled as soon as possible after the second administration of the MASQ for students who 
continued to meet the criterion at Stage 2. Students completed the DICA-IV on a laptop 
computer located in the school-based health clinic office. 

Students who received a depression diagnosis as a result of their responses to the DICA-IV 
were invited to participate in group counseling for depression. 

Results 

Stage 1 

MASQs were completed by 85.5% of the school during the homeroom administration by the 
teachers. Because this was a screening activity that would lead to further assessment of those 
students who met criterion, teachers were instructed to have students put their names on the 
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MASQ. Unfortunately, teachers were not consistent in reinforcing this request, and 16.5% of the 
MASQs were returned without names. 

All valid MASQs that were returned were used to compute the means and standard 
deviations for the Anhedonic Depression and Anxious Arousal scales. MASQs that clearly 
represented response biases (e.g., marking all I’s or all 5’s) were excluded. The mean for the 
Anhedonic Depression scale was 60.66 and the standard deviation was 12.24. The mean for the 
Anxious Arousal scale was 29.27 and the standard deviation was 1 1.05. 

Recognizing the high rate of co-occurrence between anxiety and depression, we established 
three groups using cut-offs based on scores from both the Anhedonic Depression and Anxious 
Arousal scales. The depressed group consisted of students who obtained a score one standard 
deviation above the mean on the Anhedonic Depression scale (> 72.90) and no more than half a 
standard deviation above the mean on the Anxious Arousal scale (< 34.81). The anxious group 
was composed of students who obtained a score one standard deviation above the mean on the 
Anxious Arousal scale (> 40.32) and no more than half a standard deviation above the mean on 
the Anhedonic Depression scale (< 66.78). The mixed depressed-anxious group consisted of 
students who scored one standard deviation above the mean on both scales (Anhedonic 
Depression > 72.90 and Anxious Arousal > 40.32). 

Of the students screened at Stage 1, 12.4% scored in the depressed range (i.e., were in the 
depressed or mixed groups). Students who were in the depressed or mixed depressed- anxious 
groups continued to Stage 2. 

Stage 2 

Attrition at Stage 2 was 34.3%. In other words, 65.7% of those who passed through Stage 1 
completed a second administration of the MASQ. Of these students, 60% no longer met the 
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criterion to be placed in the depressed or mixed group based on their score on the Anhedonic 
Depression scale. That meant that 4.1% of the total sample screened at Stage 1 continued to meet 
the criterion after the Stage 2 screening. 

Stage 3 

Of those who passed through Stage 2, 91 .1% continued to Stage 3 and completed the 
computerized administration of the DICA-IV. Of the students who completed the DICA-IV, 
26.8% received no DSM-IV diagnoses. The remaining students (73.2% completing Stage 3) did 
receive a depression diagnosis. This represented 3.5% of the original sample screened at Stage 1 . 

Discussion 

Generally, our results were consisted with those reported by Reynolds (1986), although we 
found fewer students scoring in the depressed range at each stage of the screening process. 
Reynolds identified 18 - 20% at Stage 1, while we identified 12.4%. Reynolds reported 12 - 
15% of the total sample continued to score in the depressed range at Stage 2; our results 
indicated that 4.1% of the total sample continued to score in the depressed range. At Stage 3, 
Reynolds reported that 7 - 12% of his total sample was identified as depressed based on a 
clinical interview. We found 3.5% of the total sample received a DSM-IV depression diagnosis 
based on the computer-administered DICA-IV, which is in line with reported prevalence rates 
(see Birmaher et al., 1996 for a review). The variations in the -percentages at the different stages 
may represent differences in the samples and/or measures used in the respective studies. Both 
samples were from the Midwest, although separated by approximately 1 5 years. In this case, 
regional differences would appear to play less of a role than they might if the samples were from 
different parts of the country. It seems more likely that the differences may be due to the 
measures employed. The MASQ was developed based on the tripartite model of anxiety and 
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depression, which is particularly sensitive to the unique features of these disorders. The 
instruments that Reynolds employed would be viewed as measures of negative affect within the 
tripartite model, and therefore would not be specific to depression. The differences between 
studies may have resulted from the fact that we used “better” measures of depression. This is 
only a hypothesis until others replicate our findings using the same measures. 

In addition to the findings from the different stages of the multiple-stage screening, we would 
like to share several practical, “lessons learned” from conducting this study. The first has to do 
with using teachers as data collectors. This has an intuitive appeal, because it means more data 
can be collected in a shorter period of time. Teachers hand out things to be completed in 
homeroom quite frequently, so we expected this to go smoothly. Although written instructions 
were provided with the MASQs, which included being sure students put their names on the 
measure, a number of teachers returned MASQs from their classes without names. This made it 
impossible to identify students who met the criterion for the second stage of screening. Our 
advise, consistent with Reynolds’ (1986) recommendation, is to provide a brief in-service for 
teachers so that everyone is on the same page when it comes to doing the large, initial Stage 1 
screening during homeroom. 

A second issue to consider when conducting a multiple-stage screening project is the cut-off 
score used to identify students who will continue through the process. We used a score of 72.90 
on the Anhedonic Depression scale, which represented one standard deviation above the mean 
for our sample. In practical terms, that meant a student had to have a score of 73 on the 
Anhedonic Depression scale to be identified. Although our approach of using one standard 
deviation above the mean is not unusual, it does mean that scores right next to the cut-off score 
are excluded. For example, there were 12 students whose score on the Anhedonic Depression 
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scale was 72. Are these students really different from those who scored 73 on the scale? 

Probably not. However, you have to establish some criterion or cut-off score. This criterion can 
be empirically based, as it was in our case, or resource based. For example, if there were limited 
resources, the cut-off score would be raised to identify fewer students. 

Cut-off scores also influenced our study in another way. In our effort to be sensitive to the 
comorbidity of anxiety and depression, we used the scores from both the Anhedonic Depression 
and Anxious Arousal scales in identifying group membership. Group membership acted as the 
identifying variable for passage through a stage. By using the criteria we used, +1 standard 
deviation (SD) and less than +.5 SD to identify depressed and anxious students on the Anhedonic 
Depression and Anxious Arousal scales, respectively, and +1 SD on both scales for the mixed 
group, we created what we call “in-betweeners.” These were students who fell between the 
criteria. For example, we had a student who had a score of 100 on the Anhedonic Depression 
scale (well over 72.90), but a score of 35 on the Anxious Arousal scale — not <_34.81 (i.e., +.5 
SD). As a result, this student would not be identified as depressed. Also, this student would not 
fall into the mixed depressed-anxious group because the score of 35 on the Anxious Arousal 
scale is less than 40.32, the +1 SD needed to fall within this group on both MASQ scales. The 
situation created by our “in-betweeners” highlights the importance of thinking through 
implications of the selection criteria established. 

A third issue had to do with what we refer to as the foibles of high school students. The 
attrition witnessed in Stage 2 of the screening process was explained in part by chronic 
absenteeism among some high school students. Repeated attempts were made to have students 
who passed through Stage 1 complete measures at Stage 2. Often, we were informed that these 
students were absent. Also, switching the second stage assessment to the auditorium, rather than 
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in the classroom, affected some students’ willingness to participate. It is one thing to do 
something everyone is doing in your homeroom. It is another thing to have to leave class or 
study hall (i.e., be singled out) to do something in the auditorium in a mixed class setting (i.e., 
with students who may be older or younger than you). 

A fourth issue had to do with the students who fell into the anxious group. Our focus was not 
on anxiety, so these students did not pass through Stage 1 unless they were in the mixed 
depressed-anxious group. This creates what we refer to as an ethical dilemma of identification. In 
other words, we identified students who may have been experiencing emotional distress, but did 
not directly act on this information. Luckily, in our situation, we were in a small, close knit 
school community, so students who experienced emotional distress were monitored through the 
mental health component of the school-based health clinic. However, whenever you “identify” 
students, whether it is in the realm of emotional or academic functioning, you should feel an 
ethical obligation to make their lives better. If you are going to conduct a multiple-stage 
screening that includes some control for comorbid conditions, you want to be sensitive to this 
issue. 

A different twist on this ethical dilemma is considering referrals from the school itself For 
example, if teachers know that you are screening for depression, they may suggest that particular 
students are depressed. These students may or may not meet the criteria that you are using in 
your screening procedure. If they do not meet the criteria, you still will want to pursue why 
particular teachers believe a student is depressed. 

Another lesson learned during this project was the strengths and weaknesses of a 
computerized diagnostic interview. An obvious strength is that the computer provides a reliable 
and economical way to interview students. Also, students generally found the computer 
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interview a novel experience. They may have even been more open about their symptoms in this 
anonymous format. The diagnostic interview did have some disadvantages. For example, the 
program did not allow for “clinical judgment.” This was most obvious when it came to the 
diagnosis of an eating disorder. There were a number of students who clearly were overweight, 
but received a diagnosis of an eating disorder, because they reported that they had lost a certain 
number of pounds. The algorithm that was programmed into the computer accepted this without 
question. This suggests the importance of reviewing student responses and confirming the 
computer-generated diagnoses, rather than blindly accepting them because they are on the 
computer printout. 

Finally, it is important to be aware of the possibility of externalizing diagnoses, even when 
you are screening for internalizing disorders. It is common to see researchers in the area of 
internalizing disorders being sensitive to the comorbidity of other internalizing disorders (e.g., 
anxiety and depression) or researchers in the area of externalizing disorders being sensitive to the 
comorbidity of other externalizing disorders (e.g., attention deficit hyperactivity disorder and 
conduct disorder). However, it is less common, but just as important, to be sensitive to the 
possibility of comorbidity of internalizing and externalizing disorders. We found that a number 
of the students who received a DSM-IV depressive disorder diagnosis as a result of their 
responses on the DICA-IV also received an externalizing diagnosis (e.g., conduct disorder, 
oppositional defiant disorder). Without recognizing the possibility of comorbid internalizing and 
externalizing disorders, it may be difficult to really know if you have “pure” groups in your 
research or why your treatment, designed for depression, is not working as expected. 

Given the many situations presented to students today, screening and early identification of 
potential difficulties are important. Screening for emotional difficulties is a secondary prevention 
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activity that clearly falls within the purview of school psychological services. In addition to 
potentially more accurate identification, multiple-stage screening is cost effective and takes 
relatively little time and effort. We highlighted some of the lessons we learned during our 
screening for depression in a high school setting. We hope that our experiences will be helpful to 
those of you considering using multiple-stage screening procedures in your settings. 
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